Skip to content

feat(ci): run multiple nightly test configurations#1102

Open
victor-dumitrescu wants to merge 1 commit into
mainfrom
vdum/more_nightly_tests
Open

feat(ci): run multiple nightly test configurations#1102
victor-dumitrescu wants to merge 1 commit into
mainfrom
vdum/more_nightly_tests

Conversation

@victor-dumitrescu

@victor-dumitrescu victor-dumitrescu commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

What

Expands the NDS Database nightly test matrix from 1 to 20 tests, with varying test parameters (operations / epoch, test cases / epoch).

Why

Expands nightly test coverage to the maximum number of tests that can actually run in parallel on GitHub runners.

Manually Testing

make all

A successful nightly workflow, with tests shortened to 10 minutes here. Each replica runs with a different seed, so test with same parameters are not redundant.

Tasks for the Author

  • Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
  • Eliminate dead code and other spurious artefacts introduced in your changes.
  • Document new public functions, methods and types.
  • Make sure the documentation for updated functions, methods, and types is correct.
  • Add tests for bugs that have been fixed.
  • Explain changes to regression test captures when applicable.
  • Write commit messages in agreement with our guidelines.
  • Self-review your changes to ensure they are high-quality.
  • Complete all of the above before assigning this MR to reviewers.

@victor-dumitrescu victor-dumitrescu marked this pull request as ready for review June 22, 2026 10:43
@github-actions

Copy link
Copy Markdown

Benchmark results for revision 7fc28d1:

Metric Duration TPS
Mean 1.51614834s 26.383
Worst 1.526051779s 26.211
Best 1.501902891s 26.633
Standard Deviation ±6.203424ms ±0.108
Full results
Run Transfers Duration TPS
1 40 1.521404871s 26.291
2 40 1.514357998s 26.414
3 40 1.517814165s 26.354
4 40 1.520481597s 26.307
5 40 1.518094679s 26.349
6 40 1.515480397s 26.394
7 40 1.515813271s 26.388
8 40 1.509677761s 26.496
9 40 1.525467944s 26.221
10 40 1.511683096s 26.461
11 40 1.526051779s 26.211
12 40 1.524856101s 26.232
13 40 1.508402017s 26.518
14 40 1.501902891s 26.633
15 40 1.512768382s 26.442
16 40 1.511824614s 26.458
17 40 1.516209567s 26.382
18 40 1.519210181s 26.329
19 40 1.509115797s 26.506
20 40 1.522349693s 26.275

Compare the results above with those for the default branch.

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.91%. Comparing base (809bddc) to head (e5a3492).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1102      +/-   ##
==========================================
- Coverage   89.92%   89.91%   -0.01%     
==========================================
  Files         135      135              
  Lines       29269    29269              
  Branches    29269    29269              
==========================================
- Hits        26320    26318       -2     
- Misses       2127     2128       +1     
- Partials      822      823       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant